Mahdi Haghifam

Email(preferred): haghifam.mahdi@gmail.com Email: mhaghifam@ttic.edu

About Me

I am a machine learning researcher. Recently, I have been especially interested in problems motivated by language models, such as distillation and inference-time algorithms. My broader work spans generalization theory, as well as the design of privacy-preserving learning algorithms and the characterization of their limits.

Recognition of my work includes a Research Fellowship at the Simons Institute (UC Berkeley), a Best Paper Award at ICML 2024, and multiple awards and scholarships from the University of Toronto for research excellence during my PhD studies.

I am currently a Research Assistant Professor at TTIC. In Spring 2026, I was a Research Fellow at the Simons Institute for the Theory of Computing at UC Berkeley. Previously, I was a Distinguished Postdoctoral Researcher at Northeastern University, working with Jonathan Ullman and Adam Smith. I completed my Ph.D. in Machine Learning (thesis) at the University of Toronto and the Vector Institute, advised by Daniel M. Roy. Before that, I received my B.Sc. and M.Sc. from Sharif University of Technology in Tehran.

I've worked in industry at Google Brain/DeepMind (with Thomas Steinke) and at ServiceNow Research (with Gintare Karolina Dziugaite). See details here.

Selected Papers

• The Distillation Game in LLMs: Adaptive Evaluations & Efficient Defenses
  Y. Allouah*, M. Haghifam*, R. Shokri, S. Koyejo
  In submission, 2026
  Summary Slides Code

  A distillation defense modifies a model’s answers at inference time, aiming to keep them useful to users while making them harder for a student model to learn from. Distillation defenses can look strong under standard fine-tuning, but this does not mean the released answers are hard to learn from. We show that weighted SFT—giving more weight to some of the same released answers—improves student accuracy by about 50% against antidistillation sampling on GSM8K and MATH, without extra teacher queries. This highlights the importance of data weighting for getting the most out of SFT, and shows that students can still learn from imperfect data. It also reveals a subtle issue in evaluation: poor student performance may reflect the training procedure, not the lack of useful learning signal. We model this interaction as a game between a teacher that must remain useful and a student that can adapt its training. This framework also gives Product-of-Experts, a cheaper distillation defense with similar robustness under adaptive evaluation.

• Towards Faithful Data Selection: Statistical Limits and Scalable Algorithms
  A. M. Inane*, M. Haghifam*, G. K. Dziugaite, I. Mitliagkas
  In submission, 2026
  Summary Code

  Data selection is often formulated as finding a small subset that leads to a low population risk. In this paper, we show that, for some problems, finding such a subset may require a validation set much larger than the training set. This motivates a shift in our view of the goal of data selection: instead of minimizing population risk, the goal is to preserve the behavior of training on the full dataset. We call this faithful data selection. This view connects different existing methods, including CRAIG, GRAD-MATCH, TracIn-style methods, and GraNd, by relating them to different parts or approximations of a common gradient-discrepancy objective. We also propose an efficient data-selection algorithm that takes into account the covariance of the gradient vectors and has provable guarantees.

• Adaptive Generate-Rank-Verify in Language Models: Inference-Time Search with Costly Verification
  S. Dughmi, M. Haghifam, Y.H. Kalayci (authors listed in alphabetic order)
  NeurIPS 2026
  Summary Slides Code

  A common paradigm for inference-time compute is to generate many candidate responses, rank them with a cheap reward model, and pass the top-ranked ones to an expensive verifier. Since verification is often far more costly than generation and scoring, the policy faces a per-prompt compute allocation problem. We formalize this as active search: a sequential decision problem with a cheap reward-observation cost and an expensive verification cost. Our ADAP algorithm achieves a constant-factor competitive ratio against the distribution-aware optimum under a natural monotonicity assumption on the reward signal, and on HMMT math and LiveCodeBench coding it reaches 100% success rate at roughly 3x to 5x lower mean cost than the best fixed generate-and-verify policy. We also provide learning-theoretic view of this paradigm.

• The Sample Complexity of Membership Inference and Privacy Auditing
  M. Haghifam, A. Smith, J. Ullman
  FOCS 2026 (Invited to TPDP 2026 Special Section of Journal of Privacy and Confidentiality)
  Summary Slides

  We characterize the sample complexity of membership inference in the fundamental setting of Gaussian mean estimation. The main finding is that standard empirical attacks using O(n), where n is the number of training samples, reference samples can substantially underestimate the true privacy risk, since matching the fully-informed (Bayes-optimal) adversary requires far more reference data than the size of the training set itself, i.e., n^2.

• Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorization
  I. Attias, G. K. Dziugaite, M. Haghifam, R. Livni, D. M. Roy (authors listed in alphabetic order)
  ICML 2024 (Oral, Best Paper Award: Top 10 of 10,000 submissions)
  Summary Talk

  We establish a strict tradeoff between excess loss and the conditional mutual information any algorithm extracts from its training data in stochastic convex optimization. Achieving small error necessarily requires revealing substantial information about individual samples, which yields lower bounds on memorization and a direct connection to vulnerability against membership inference attacks.

• Faster Differentially Private Convex Optimization via Second-Order Methods
  A. Ganesh, M. Haghifam, T. Steinke, A. Thakurta (authors listed in alphabetic order)
  NeurIPS 2023
  Summary Code

  We show that second-order information can be used effectively under differential privacy, despite the added noise. Our algorithm, a private variant of the regularized cubic Newton method, adapts to the local shape of the loss and achieves the optimal excess loss for DP-ERM while running 10 to 40 times faster in wall-clock time than DP-SGD at comparable accuracy and privacy.

• Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithms
  M. Haghifam, J. Negrea, A. Khisti, D. M. Roy, G. K. Dziugaite
  NeurIPS 2020
  Summary Talk Code

  We give data-dependent generalization bounds for noisy iterative algorithms such as SGLD using the conditional mutual information framework. Unlike worst-case Lipschitz- or norm-based bounds, the resulting estimates depend on the algorithm's actual trajectory on the training data and remain non-vacuous in modern overparameterized settings.

Industry Internship Experience

Professional Service

Area Chair: ALT (2026), ICLR (2027), NeurIPS (2026) (Top AC award 2026, Top Reviewer Award 2021)  ·  Senior Program Committee: ALT (2027)  ·  Program Committee: SaTML (2024, 2026), COLT (2026)

Contact Me!

Feel free to reach out if you'd like to discuss research ideas. Also, I'm happy to offer guidance and support to those applying to graduate programs, especially individuals who might not typically have access to such assistance